Method and Apparatus for Light Field Generation

ABSTRACT

A nanophotonic phased array is configured to generate dynamic three-dimensional imagery when employed as an oscillatory beam-steering device. A scanning nanophotonic phased array generates programmable light fields. That is, a phased array generates reconfigurable light fields when controlled to perform an angular scan of incident illumination synchronized with respect to modulation of the incident illumination.

The present application claims the benefit of U.S. Provisional PatentApplication No. 62/450,855, filed Jan. 26, 2017, titled “Method andApparatus for Light Field Generation,” the entire contents of which arehereby incorporated by reference, for all purposes.

TECHNICAL FIELD

The present invention relates to the use of an optical array ofantennas, and more particularly to nanophotonic antennas in a phasedarray associated with a phase shifter, as a light field generator forsuch uses that include 3-D display and beam scanning for electronicdisplay among others.

BACKGROUND ART

Autostereoscopic 3-D displays generate imagery visible to the unaidedeye. The specific characteristics of the imagery depend on theoperational mechanisms of the display device, but their propertiesusually include: (1) appearance in front of, behind, or straddling thedisplay, (2) visibility as three-dimensional within a range of angles ordistances from the display, (3) having a perceived spatial resolution,often specified at a surface of greatest detail (e.g. the displaysurface if one exists), (4) responsiveness to time-varying input, e.g.capable of displaying dynamic rather than static imagery, and (5) forimagery comprised of discrete perspective views, an angular view densitywhich, ideally, is chosen so that the reconstructed 3-D scene does notexhibit visible “jumping” from view to view during user head motion.

For context, a typical 3-D display system performs the steps of:

(a) capturing or rendering information representative of a 3-D scene andstoring it in a memory subsystem as image data;

(b) providing subsets of the image data to a projection engine of thedisplay; and

(c) optically presenting the image data as to project a 3-D image (knownas reconstruction or replay).

Examples of typical 3-D displays, and approaches for performing (a) and(b), are detailed in the following references:

-   -   Halle, “Autostereoscopic displays and computer graphics,”        SIGGRAPH Comput. Graph., pp. 58-62 (May, 1997);    -   Holliman, et al., “Three-Dimensional Displays: A Review and        Applications Analysis,” IEEE Trans. Broadcasting, pp. 362-71        (2011);    -   Chun, et al., “Spatial 3D Infrastructure: display-independent        software framework, high-speed rendering electronics, and        several new displays,” in SPIE Stereoscopic Displays and Virtual        Reality Systems XII, (ed. Woods et al.), Proc. SPIE-IS&T        Electronic Imaging, SPIE, vol. 5664, pp. 302-312 (2005); and    -   J. Geng, “Three-dimensional display technologies,” Adv. Opt.        Photon., vol. 5, pp. 456-535, (2013); and    -   Lucente, “Computational holographic bandwidth compression,” IBM        Systems Journal, vol. 35, pp. 349-65 (1996).        All of the foregoing references are incorporated herein by        reference.

Creating 3-D imagery by projecting an image sequence synchronized to theoscillatory motion of an opto-mechanical beam-steering device, such asthe lenticulars described in U.S. Pat. No. 7,864,419, is noisy,difficult to construct at scales greater than 30 cm×30 cm, and have alimited field of view. U.S. Pat. No. 7,864,419, titled “Optical ScanningAssembly,” (hereinafter “Cossairt '419”), is incorporated herein byreference. Moreover, these devices are difficult to operate in atwo-axis (full parallax) scan mode because at least one mechanical axismust run at a very high frequency.

Creating static 2-D imagery in the far field has been demonstrated witha “pre-programmed” nanophotonic phased array using physically basedinterference modelling that requires the computation of potentiallytrillions of delay states to create an image of viewable size (See U.S.Pat. No. 8,988,754, and Sun, et al., “Large-scale nanophotonic phasedarray,” Nature, vol. 493, pp. 195-99, (2013), the entire content of eachof which is hereby incorporated by reference herein for all that itdiscloses). However, the generation of dynamic (video) imagery using theGerchberg-Saxton algorithm as reported is computationally expensive andrequires orders of magnitude more delay-line controllers than theinvention described here.

No electro-holographic or diffractive display of practical utility at avariety of scales has yet been demonstrated in the prior art. One reasonfor this is that the optical modulator is either too slow, or has pixelsthat are too large compared to the wavelength of light (resulting inimagery that either restricts head motion or requires a large outputlens), or is run in a diffractive mode other than phased-array beamsteering, which requires a complex scheme for asserting phase delays.

With rare exception, no autostereoscopic display technology has been ofsufficient quality and utility to be widely adopted. Today's volumetric,lenticular, multi-projector, and scanned-view 3-D displays have beensome combination of: unsuitably large for packaging into tablet ortelevision form factors, uncomfortably narrow viewing angle, low imageresolution at the display surface and throughout the reconstructed imagevolume, and computationally intensive.

Within the field of 3-D display, it is well known that 3-D imagery canbe generated when light, representative of regions of a scene from acollection of viewpoints, is scanned in several directions towards aviewing region within the integration period of the human eye. Thisarrangement enables each eye of a viewer to potentially see a differentimage, which is a stereoscopic depth cue. For suitably broad fields ofview, one or more simultaneous users can place their heads in differentlocations, inspecting a scene from various points of view.

Time-multiplexed autostereoscopic displays place demands on thefrequency with which a set of light-transmitting regions must modulate,and on the number of such modulators. In one example, a 20,000frame-per-second digital projector casts light onto a 30 cm×30 cmbeam-steering array that performs oscillatory horizontal scanning at 50Hz. In this case, the 3-D image is decomposed into 200 two-dimensionalviews, and the set of views are projected during each horizontal sweepof the scanner every 1/100 sec ( 1/100+ 1/100= 1/50 sec=50 Hz).Therefore, a 100×200=20,000 frame-per-second image source is required.

Workers in the field of 3-D display have experimented with various agilebeam steering devices for 3-D display, such as two lenticular arraysundergoing relative vibratory motion, as described in Cossairt '419.Systems of this type have suffered from drawbacks including: narrowhorizontal and/or vertical field of view, insufficient angularresolution, and acoustically noisy operation.

SUMMARY OF THE EMBODIMENTS

In accordance with an embodiment of the present invention, a radiationprojector is provided that has a plurality of nanophotonic antennasconfigured to emit electromagnetic radiation. The radiation projectoralso has phase delay elements, each one characterized, at any particularmoment, by a phase delay. At least one phase delay element is associatedwith at least one of the plurality of nanophotonic antennas. Theradiation projector also has a control signal generator configured togenerate a control signal associated with the plurality of delayelements, wherein the control signal is further configured torecurrently update phase delays of the plurality of phase delay elementsin such a manner as to cause the electromagnetic radiation tosubstantially span at least one contiguous solid angle. In certainembodiments of the invention output of the antennas may generate a threedimensional light field of imagery.

A nanophotonic phased array can be configured to generate dynamicthree-dimensional imagery in an efficient manner when employed as anoscillatory beam-steering device. A scanning nanophotonic phased arraycan generate programmable light fields. That is, a phased array cangenerate reconfigurable light fields if it is controlled to perform anangular scan of in-coupled or incident illumination and is synchronizedwith respect to the modulation of the incident illumination. This willbe explained in the context of 3-D display. The system may use anoptical array of antennas, configured as an array of nanophotonicantennas, to generate arbitrary light fields in applications including3-D display. The array of nanophotonic antennas are assembled in anarray of arrays, and operated in a phased array configuration to performbeam-scanning for electronic display.

Other techniques of electro-holographic display that may be applied tonanophotonic antenna arrays to generate 3-D imagery are also disclosed.For example, in addition to using the antennas as directional elements(direls) that perform a horizontal or two-axis periodic sweep of space,the antennas of the nanophotonic array can be operated as holographicpixels (hogels), each projecting an illumination cone representative ofthe appearance of a scene from a collection of viewing angles.Additionally, the nanophotonic phased array can be operated to produce“wafels,” by imposing a desired curvature on each piecewise contributionof a reconstructed light field's wavefront.

In other embodiments of the invention, a MEMS phase shifter is used forshifting the phase of the illuminated signal to direct theelectromagnetic radiation to the proper antenna at the proper angle.

The radiation projector may also include a modulator for receiving datarepresentative of a three dimensional scene and producing a plurality oftime-varying illumination patterns. In certain embodiments, the scenemay be scanned from several different directions and projected inseveral different directions so that a view can move their head and lookaround the scene.

In accordance with other embodiments of the invention, theelectromagnetic radiation emitted by the antennas may be visible light.The radiation projector may also include the plurality of nanophotonicantennas coupled to the modulator for dividing the time-varyingillumination patterns into a plurality of paths, wherein a path isassociated with each antenna within the array.

In other embodiments of the invention, the plurality of delay elementsare a plurality of phase shifters each associated with an antenna forshifting the phase of the time-varying illumination patternscollectively so that the patterns are modulated in synchrony for each ofseveral directions.

The phase delay elements may shift the phase in both a horizontal and avertical direction. The phase shifters may be associated with an arrayof nanophotonic antennas that represent a single pixel and the phaseshifters for the single pixel receive a phase control signal in the xdirection and in they direction. In such a configuration only two phaseshift control values are needed to steer a radiation pattern from theantennas in the array.

When the nanophotonic array is arranged in an array of arrays, eacharray can represent a single pixel. In addition to representing a pixelthe radiation pattern may represent a hogel, a direl, or a wafel.

The radiation projector according may also include a controller forproviding phase shift control signals to each of the phase delayelements to provide a sequence of video images.

In certain embodiments of the invention, the plurality of phase delayelements is a MEMS phase shifter that mechanically alters phase basedupon movement of a phase actuator. The phase actuator may be a movablemembrane.

The radiation projector may also include a database containing datarepresentative of the three dimensional scene scanned from severaldifferent directions.

In accordance with further embodiments of the invention, thenanophotonic antennas may be controlled with a first control signal tosteer the emitted output radiation in a first direction. In otherembodiments of the invention the nanophotonic antennas are controlledwith a second control signal to steer the emitted output radiation in asecond direction. The plurality of nanophotonic antennas may be arrangedin an array and the array may be arranged on a two dimensional surface.

In another embodiment of the invention the radiation projector includesa lens and a plurality of interconnected switches having an input and anoutput wherein the input receives a time-varying illumination patternand the time-varying illumination pattern is synchronized with controlsignals to the plurality of switches allowing the time-varying patternto be directed in a desired direction through the output of the switchesand through the lens. In certain embodiments the plurality ofinterconnected switches have a plurality of outputs that defines apixel, wherein the direction of the emitted time-varying illuminationpattern is dependent on the state of the switches.

In yet another embodiment of the invention for a radiation projector,the radiation projector includes a lenticular lens, an input forreceiving a time-varying illumination pattern, a nanophotonic arrayhaving a plurality of outputs and a filter for directing thetime-varying illumination pattern to a particular output of thenanophotonic array so that the time-varying illumination from the outputis directed to the lenticular lens.

In accordance with other aspects of the present invention, methods areprovided for generating a three-dimensional radiation pattern. Themethods have steps of:

-   -   receiving data from a data store that defines a three        dimensional image;    -   converting the data into a time-varying illumination pattern and        providing the time-varying illumination pattern to an input of a        nanophotonic array, wherein the nanophotonic array includes a        plurality of antennas and the antennas emit electromagnetic        radiation; and    -   delaying the electromagnetic radiation with a plurality of phase        delay elements using a control signal, at least one phase delay        element associated with at least one of the plurality of        antennas, wherein the control signal is periodic.

Corresponding methods are provided, in accordance with furtherembodiments of the invention, wherein the electromagnetic radiation isswitched with a plurality of switching elements using a control signal,at least one switching element associated with at least one of theplurality of antennas.

In yet another embodiment of the invention, for a radiation projector,the radiation projector steers a phased array by changing the wavelengthof the input signal. For example, a radiation projector includes aplurality of optical couplers, each optical coupler transmitting a firstportion of a lightwave incident thereupon and radiating a second portionof the lightwave, the lightwave characterized at any point by awavelength-dependent phase. The radiation projector may also include awaveguide for transmitting the lightwave successively to a succession ofthe plurality of optical couplers in such a manner that thewavelength-dependent phase varies between successive optical couplers bya fixed wavelength-dependent increment.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing features of embodiments will be more readily understood byreference to the following detailed description, taken with reference tothe accompanying drawings, in which:

FIG. 1 is an exemplary system illustrating the reconstruction of a 3-Dscene by a 3-D display system;

FIG. 2 illustrates a top view of the 3-D display system in operation;

FIG. 3A illustrates a top view of a 3-D display system including each ofthe components of such a system; FIGS. 3B and 3C schematically depictacquisition and display, respectively, of scene data, in accordance withan embodiment of the present invention;

FIGS. 4A-4D shows different patterns that may be generated by a tile(nanophotonic phased array with a plurality of phase delay elements)wherein each tile is analogous to a pixel in 2-D display and any of:“hogel,” (FIG. 4B) “direl,” (FIG. 4C) “wafel,” (FIG. 4D) may also beproduced by the tile;

FIG. 5 shows an exemplary tiled array of phased arrays;

FIG. 6 is a schematic diagram plan view of a phased array 100 of opticalcouplers, represented by circles, arranged in an H-tree;

FIG. 7 is a schematic diagram plan view of a portion of the phasedarray;

FIG. 8 is a schematic diagram plan view of a dynamically tunable opticaldelay line;

FIG. 9 shows a MEMS phase shifter;

FIG. 10 shows an alternative embodiment wherein phase is controlled in atwo-axis system for steering the beam;

FIG. 11 is a representation of how the initial data is derived andstored in a storage space prior to be retrieved for display using thenanophotonic array embodiments;

FIG. 12 represents the various directions that the outputs of a tile canbe directed including both parallel directions and convergingdirections;

FIG. 13 illustrates a single tile of a display array, configured as anelement of a spatially multiplexed display, using active steering by anetwork of switches;

FIG. 14 illustrates an alternative steering technique that employspassive wavelength-selective switching, also as a tile of a spatiallymultiplexed display using a lenticular lens;

FIGS. 15 and 16A-16B show methods based on nanophotonic phased arrays tosteer the light in each tile: FIG. 15 shows an embodiment of theinvention in which active steering is performed by phaseshifters/modulators, and FIG. 16A shows an embodiment of the inventionthat includes passive beam steering where the beam is steered accordingto wavelength. FIG. 16B shows an exemplary tile in which the lightfollows the serpentine path as shown in FIG. 16A, in accordance with anembodiment of the present invention.

DETAILED DESCRIPTION OF SPECIFIC EMBODIMENTS

Embodiments of the invention solve the problems of the prior art byutilizing nanophotonic phased array “tiles,” assembled into tile groupsof arbitrary size, in a variety of periodic or aperiodic beam-steeringmodes, while input illumination is modulated in synchrony with eachtile's beam direction and the corresponding elements of a database ormemory representative of a visual scene to be reconstructed.

Definitions

The word “arbitrary,” is used herein to refer to a value of a parameter(such as the size of a component) that may be specified by a designer ofa system as a matter of design choice, or that is presented by thesystem to be elected by a user, as a matter of convenience.

A “modal index,” as the term is used herein, refers, as normative in theart, to the effective refractive index of a waveguide mediumparticularized to a specified mode of propagation in the medium.

A control signal generator includes any circuitry, known in the art orsubsequently developed, that applies a control signal to a succession ofcontrol elements, whether in parallel, in whole or in part, orsuccessively, or in some combination of parallel and serial application.The control signal generator may be implemented in whole, or in part, inanalog or digital embodiments.

“Recurrently” means beginning at successive instants of time, and mayinclude periodic behavior, but is not so limited, as it may includepatterns that are not identical from sweep to sweep, and that may, infact, be random or quasi-random.

“Update a phase delay” means to apply a control signal to each of thesuccessive phase delay elements. Updating a phase delay may includemoving the control signal to successive sets of phase delay elements.

“Substantially span” means encompassing a region (of solid angle, forexample) so that it is perceived by the eye as covering that region.

With the appropriate synchronization of scanning and amplitudemodulation, users will perceive imagery due to the persistence of visionover the period of the system's scanning action.

An additional embodiment of the invention employs MEMS phase shifters inplace of other phase delay elements. The MEMS phase shifters employmechanical principles for shifting the phase of the optical signal. Onesuch embodiment, includes a moveable structure that physically interactswith the waveguide to change the optical path and thus, the phase of theoptical signal. In another embodiment, a membrane is moved closer orfurther away from the waveguide altering the evanescent fields thatextend from the waveguide and changing the modal index of the waveguidesystem.

Another embodiment of the invention uses an antenna array to create 3-Dimages, for example in analogy to lenticular arrays and integralphotography.

This method has several benefits over other scanned-viewpointautostereoscopic displays, including: a thin form factor suitable forpackaging into consumer electronic devices; the ability to be tiled withminimally-visible seams between tiles; having a high spatial resolutionat the display surface; and compatibility with semiconductor laserillumination offers high switching speeds between views, therebypermitting several views per pupil area. This elicits an accommodativeresponse (i.e., refocusing) in the viewer due to the realistic projectedimagery.

Embodiments of the invention are also directed to using an optical arrayof nanophotonic antennas (or an array of arrays) to generate arbitrarylight fields in applications including 3-D displays. Embodiments of theinvention are useful generally for projecting light fields. Light fieldshave applications in three-dimensional displays with or without opticalheadgear for the viewer, such that the images can be projected in freespace. Embodiments of the inventive system may provide opticalactivation of biological tissue, such as optogenetics and neurogenetics.In such embodiments, because of the small scale of the arrays, theoptical signal (i.e. light) can be directed to a plurality of neurons,so that multiple neurons can be activated simultaneously. In otherembodiments, the nanophotonic arrays can be used for free-spacecommunications, for projecting a beam and steering the beam, wherein 1and 0 (or n-ary values) can be represented by different phases and thebeam can be steered toward a receiver. For example, the optical beam maybe steered toward a moving satellite. In another embodiment of theinvention, the nanophotonic arrays may be used for perspective-correctoptical cloaking. In such an embodiment, a vehicle, such as a tank,could be covered with nanophotonic arrays and the beamed imagery canchange with the viewing angle. This would provide for a much more realpresentation of camouflage, as the camouflage would change with theperspective of the aerial viewer. In yet another embodiment of theinvention, methods in accordance with the invention may be used for 3-Dprinting in which the beam steered light can cause a printing materialto harden.

The fact that embodiments of the present invention employ diffraction todefine beam characteristics provides an unprecedented degree of lightfield reconstruction flexibility. It is essentially a dynamicallyaddressable hologram. Each tile can be directed to run in a variety ofmodes that “dial in” the realism of the reconstructed scene (by default,as directional elements, which are piecewise approximators to sphericalwave front sections).

Embodiments of the invention can be the optical engine ofelectro-holographic displays, such as: a desktop 3-D computer display, ahead-worn near-eye virtual reality/augmented reality/mixed realitydisplay, a virtual sand table, or the walls of a room creating immersiveimagery. Applications of such displays include: battlefieldvisualization, interventional medical imaging for procedure planning andguidance, molecular visualization and entertainment. FIG. 1 shows anexemplary system illustrating the reconstruction of a 3-D scene (10) bya 3-D display system 20. The 3-D display system has an image projectionsurface (21) composed of emissive regions 51 having horizontal andvertical angular emissive extents, yielding a viewing zone 50. Anobserver (40, 41—represented by his/her eyes) is able to see the 3-Dscene 10 processed by processor 30 when situated within the viewing zone50. A viewer outside the viewing zone who is not gazing at the imageprojection surface, such as a third observer 42, will not see a 3-Dimage because light representative of the 3-D scene will not enter thepupils of the third observer. As noted, the display emits light suchthat each eye sees a potentially different image. The emissive regionsoutput light with an intensity and direction (or other emissive profile,such as a curved wave front) with time-varying properties.

FIG. 2 illustrates a top view of the 3-D display system in operation.The observer (40) with left (40L) and right (40R) eyes will perceive a3-D image when the image projection surface emits illuminationrepresentative of two perspective viewpoints of the 3-D scene (100,110).

FIG. 3A illustrates a top view of a 3-D display system including each ofthe components of such a system. A source 61 of 3-D data such as adatabase or storage device (and otherwise referred to herein as memory61) provides information representative of a 3-D scene to a displaycontroller 62. The data stored in the storage device may be capturedphotographs from a 360-degree view of an object or the data may be threedimensional computer generated data. The photographs do not need to be a360-degree view and might be just a small number (as in the range of2°-100°) from different perspectives of a camera moving along a lineartrack. FIG. 3B shows the acquisition of 3D scene data, designatedgenerally by numeral 22, for later presentation and viewing by anobserver. Cameras 23A, 23B and 23C are positioned at different locationsto capture scene 22. As shown, there is a left view camera 23A, a centerview camera, and a right view camera that each captures an imagesimultaneously. This image data is passed through video processor andstored in memory 61 for later retrieval and is indexed with respect totime (i.e., relative to a clock signal CLK).

FIG. 11 is a representation of how the initial data is derived andstored in a storage space prior to being retrieved for display using thenanophotonic array embodiments described in the present disclosure. Asshown, a car 1200 is the desired scene to be projected by an embodimentof the present system. A movable camera or a series of stationarycameras take pictures of the car from different angles. These camerascan be real or synthetically generated cameras (i.e., for computergraphics). Each camera 1210, 1220 . . . 1230, 1240 captures a separateview of the flowerpot from a left-most view 1250 to a right-most view1260. This data is then combined together to form a 3D data set usingtechniques known to one of ordinary skill in the art.

During display of the source data stored in memory 61, the displaycontroller 62 provides time-varying illumination patterns along one ormore connections 63 to a tiled array of phased arrays 60 and may also bereferred to as a “modulator.” Thus, the display controller 62 convertsthe three dimensional (3-D) data into the time varying illuminationpatterns.

One or more connective elements 63 (also referred to herein as“connections” 63) may be electro-magnetic or optical waveguides, forexample, that separate the 3-D data scene into a plurality of lines,wherein each line may be associated with a separate pixel for display.

A single phased array 102 is illustrated in context. The design of thesource 61 of 3-D data and the display controller 62 are well-known topractitioners in the art of 3-D display systems engineering. The displayof scene data is shown in FIG. 3C. Display controller 62 includes anarray of illuminators 65 (e.g., laser diodes coupled to fiber, otherwisereferred to, for heuristic convenience, as laser diodes 65) and acontroller block 67 that generically handles accepting an input clocksignal, CLK. The controller block 67 cycles through video data addresscounter 68, and in synchrony with clock signal CLK provides the phasedelays (e.g., ϕ₁ and ϕ₂), the address and optionally some controlsignals to the laser diodes. As the device is turned on, the addresscounter is set to “0” and ϕ₁ and ϕ₂ are set to the phases correspondingto steering light to the left. The image data of the left view arrivesat the block of illuminators, and goes to the array of the phased array.Then the center and right views are also sent. This process cyclesthrough for all of the data stored in memory. The time-varying emissiveproperties of each emissive region is controlled by a display controller69. Display controller 69 provides control signals that include, but arenot limited to: time-varying illumination intensity for each wavelengthband(s) of each emissive region, and control regarding the diffractivefunction of each emissive region. In one example, the diffractivefunction of the emissive regions is a phased-array beam steeringfunction.

In this first example, each emissive region is a phased array and isalternatively referred to as a “tile” 51. FIG. 4A-D shows differentpatterns that may be generated by a tile wherein each tile 51 isanalogous to a pixel in 2-D display (FIG. 4A) vernacular and any of:“hogel,” (FIG. 4B) “direl,” (FIG. 4C) “wafel,” (FIG. 4D) or otherdiffractive or beam-steering schemes in 3-D display vernacular. Theterms hogel, direl, and wafel are familiar to engineers in the field of3-D display architecture and are described, in addition to otherarbitrary wavefront segmented displays, in Smithwick et al.,“Interactive holographic stereograms with accommodation cues,” Proc.SPIE 7619, Practical Holography XXIV: Materials and Applications, 761903(Feb. 10, 2010); doi: 10.1117/12.840526 and in Plesniak et al.,“Reconfigurable image surface holograms,” U.S. Pat. No. 6,927,886, bothof which are incorporated herein by reference in their entirety.

An exemplary tiled array 60 of phased arrays 102 is now described withreference to FIG. 5. Each tile 51 is an array of nanoantennas capable ofbeing driven in a variety of space and time patterns, such as a phasedarray described above. Although a 6×6 tile array is shown for reference,the size, number, and arrangement of the tiles is determined as a matterof design choice to suit an intended application. For example, a desktopholographic video display may measure 600 mm×300 mm, and each tile wouldoccupy 0.5 mm×0.5 mm. Each tile would direct modulated illumination in atime-varying angular scan subtending a half angle of 45 degreeshorizontally to either side of the normal vector of the display, and ahalf angle of 20 degrees vertically. For a perspective projectionangular density of one view per degree, the system wouldtime-sequentially direct 2-D fields of modulated light to((45*2)*(20*2))=3,600 directions per reconstructed scene, at a rate of60 Hz. This requires a modulation frequency of 3,600 directions/scene*60scenes/second=216,000 directions/second.

Operation in a horizontal parallax only (HPO) configuration reduces thescan requirements to (45*2)=90 directions per reconstructed scene, at arate of 60 Hz, which equals 5,400 directions/second. To permit verticalhead motion, the tiles must have a vertical emission component, e.g., byarrangement as a horizontally-oriented linear array; else the outputshould be vertically diffused, such as by an overlaid vertical diffuser,available from Luminit, LLC of Torrance, Calif., or by asserting avertical beam broadening component to the appropriate delay lines ofeach tile.

Continuing the example of a display formed of 0.5 mm×0.5 mm tiles, thereis a tradeoff between the image quality, tile size (phased array area),and spacing between nanoantennas. As one illustrative example, tile 51could have an array of 100×100 antennas, with a spatial period of 500nanometers. One-dimensional arrays would have 100 antennas. Tiles couldhave far fewer antennas, such as an array of 10×10, or far greaternumbers of antennas, such as an array of 1,000×1,000 or 10,000×10,000.Likewise, the spatial period of antenna placement can vary fromsub-wavelength to many wavelengths.

A typical tile 102 is described below with reference to FIG. 6. FIG. 7provides additional detail of a typical tile, and FIG. 8 shows theantenna structure. A more comprehensive discussion of phased arrays canbe found in US Published Patent Application No. 2016/0245895, entitled“Zero Optical Path Difference Phased Array,” which is incorporatedherein by reference in its entirety. Provided below is a briefdescription of the phased arrays and the individual components of phasedarrays. This description should not be viewed as limiting, but rather asan exemplary description of one type of phased array.

FIG. 6 is a schematic diagram plan view of a phased array 100 of opticalcouplers, represented by circles, arranged in an H-tree 102, accordingto an embodiment of the present invention. The optical couplers,exemplified by optical couplers 104, 106, 108 and 110, are connected toleaves of the H-tree 102. Each optical coupler 104 may be referred toherein as an “optical antenna.” Lines in the H-tree, exemplified bylines 112, 114 and 116, represent optical waveguides or other opticalfeedlines. The optical waveguides 112-116 meet at opticalsplitters/combiners, represented by junctions 118, 120 and 122 of thelines 112-116. For example, the optical waveguides 112 and 114connecting optical couplers 104 and 106 meet at an opticalsplitter/combiner 118.

Optical waveguides 112 and 114 are of equal lengths. Similarly, otherpairs of optical waveguides 112-116 that meet at common junctions are ofequal lengths. The direction of combination alternates (left-right,up-down) between successive optical splitters/combiners 118-122 toensure each signal combination occurs in phase. The resulting phasedarray 100 operates over a broad range of wavelengths. The entire phasedarray 100 is fed by an optical waveguide 124, which is referred toherein as a “root” of the H-tree.

FIG. 7 is a schematic diagram plan view of a portion of the phased array100. The optical waveguides 112, 114 and 116 include respectiveexemplary trimming portions 500, 502 and 504. The trimming portions 500and 502 are disposed in the light paths of optical couplers 104 and 106closer to the optical couplers 104 and 106 than any opticalcombiners/splitters 118 or 120. Additional trimming portions, such astrimming portions 504, 506, 508 and 510, may be disposed in otheroptical waveguides, further from the optical couplers 104 and 106.

The phased array 100 also includes a dynamically tunable optical delayline for each optical coupler 104-110, as exemplified by dynamicallytunable optical delay lines 512, 514, 516 and 518. Each dynamicallytunable optical delay line is disposed in a respective optical path ofthe corresponding optical coupler 104-110. FIG. 8 is a schematic diagramplan view of a dynamically tunable optical delay line 700 feeding acompact grating 702 optical coupler. Lengths of two sections 704 and 706of the dynamically tunable optical delay line 700 may be temporarilyadjusted by varying amounts of heat generated by two heaters 708 and 710that are fabricated in the substrate 200. The amount of heat generatedby each heater 708-710 may be controlled by a processor (not shown)executing instructions stored in a memory to perform processes thatsteer the phased array 100. Thus, each dynamically tunable optical delayline includes a thermally phase-tunable optical delay line.“Temporarily” means not permanently, i.e., for a finite duration oftime, wherein the duration is not necessarily predetermined. Forexample, after the heaters 708 and 710 cease generating heat, the twosections 704 and 706 of the dynamically tunable optical delay line 700return to their respective earlier lengths, or at least nearly so. Itshould be recognized that other tunable optical delay lines may be usedthat do not require heat for tuning, rather the delay may undergo phaseshifting via a MEMS phase shifter as described below, for example.

Dynamically tuning the tunable optical delay lines 512-518 controls thereal-time phase of each optical coupler 104-110 of the phased array 100.Dynamically tuning the tunable optical delay lines 512-518 makestemporary changes to the amount of delay incurred by optical signalstraversing the corresponding optical coupler 104-110. The amount ofdelay can be changed relatively quickly, thus the dynamically tunableoptical delay lines 512-518 may be dynamically tuned to electronicallysteer the phased array 100. Instead of, or in addition to, makingpermanent changes to the trimming sections, the trimming sections and/orthe tunable optical delay lines 512-518 may be temporarily changed tocompensate for fabrication non-idealities.

An alternative way to control the phased array 100 uses MEMS phaseshifters. A MEMS device can be used to affect a path length or a phasechange in an optical waveguide. Such a MEMS device is shown in FIG. 9,where a membrane 1010 is moved closer or further from a waveguide 1000.Because the mode in the waveguide 1000 has evanescent fields extendingto the membrane 1010, the movement of the membrane 1010 changes themodal index of the waveguide system. The MEMS element(s) can be movedalong a variety of locations rather than just STATE A and STATE B, suchthat it is nearly infinitely adjustable.

Alternatively, as now described with reference to FIG. 10, a membrane1005 (shown in FIG. 9) above waveguide 1130 can be moved laterally tocover more or less of the waveguide 1130. A two-axis steering mechanismis illustrated (for example, horizontal and vertical). To steer a singlebeam to an antenna structure 1120, only two categories of phase shiftneed to be asserted: Φ₁ 1100 for horizontal scanning and Φ₂ 1110 forvertical scanning. Alternatively, each phase could be set independently,providing a completely arbitrary phase hologram.

A further alternative (not shown) is to place a free-standing movableobject laterally to the waveguide. This object, such as a wire-likeobject, is moved closer or further from the waveguide, again affectingthe evanescent field extending (this time laterally) to the mainwaveguide.

The advantage of such MEMS systems is that they are compatible with awide range of optical materials, enabling this technology to be used fora wide range of wavelengths extending from the mid-IR to UV (includingthe important visible wavelengths in-between).

If it is desired to operate the phased arrays in the visible spectrum,the appropriate materials should be used. The passive waveguides can bemade of a variety of well-known materials, preferably silicon nitride,because of its compatibility with CMOS fabrication processes and highindex contrast. Alternative materials for passive waveguides includedoped silica or polymers. Active materials include GaN, AlN, and certainpolymers. If desired, a blue/UV laser can be fabricated in GaN.

Continuing the description of the first example, as shown in FIGS.1-3A-3C, the tiles act to reconstruct a 3-D scene in the followingsequence. See the left side of FIG. 12. An array 1310 of light sources1301 is controlled by the connections 63 (shown in FIG. 3A) to emitlight representative of a first viewpoint 1302 of a 3-D scene, and thearray of tiles are controlled so that their far field radiation patternsare tilted to an angle that corresponds to the first viewpoint 1302 of a3-D scene. Next, the array of light sources are controlled to emit lightrepresentative of a second viewpoint 1303 of the 3-D scene, while thearray of tiles are controlled so that their far field radiation patternsare tilted to the angle to the second viewpoint 1303, and so on. Thetechniques used to assert the scan angle e.g. within each tile,thermally controlled delay lines or phase shifters (e.g. MEMS phaseshifters) assert the exit angle of each tile's outgoing light.

Several variations of the scheme taught here are now discussed, in thefollowing categories: tiling geometries, scan directions, illumination,the diffractive function of each tile, and the “coherence” or “jointpurpose” of display tiles.

Tiling Geometries

The display can be formed of tiles in a variety of arrangements: linear(1-D), meandering linear, non-abutting, abutting, surface (2-D), orother arrangements. (The word “areal” may be used herein adjectivallywith a meaning synonymous with that of “surface.”)

Scan Directions

For example, tiles can steer light horizontally, vertically, in a 2-Draster scan, or in a random or pseudo-random pattern. Referring furtherto FIG. 12, at any specified instant, the phased array tiles can steerlight in the same direction 1302, 1303, in different directions 1304-06,or in connected or disconnected groups. For example: The entire arraycan steer light at the same angle 1302, 1303, with respect to the lineor surface of the tiles. The constituent beams of a 3-D scene can bederived from a variety of samplings through the data descriptive of the4-D light field. For example, the first example described herein, asshown in FIG. 3A-3C, utilizes a scanned parallel pencil of rays.Alternatively, the array can steer light collectively towards a locus inspace 1314, 1315, 1316, and move that locus during scan. (Compare, forexample, the two scan patterns of FIG. 12).

As a further alternative, each tile can scan in a pseudo-random pattern.Regardless, if the display output is intended for viewing by a human,every desired scan angle should be projected by each tile over theintegration period of the eye; Different tiles, or different collectionsof tiles, can steer light in one or more directions. For example: theleft display half can perform a horizontal scan while the right displayhalf can perform a vertical scan. Tiles can be arranged in arrays thatare 1-D (linear, or a meandering line or curve), or a 2-D surface (aplane, a curved surface such as a concave hemisphere, orscattered/disconnected). The number of antennas per tile can beradically decreased to a linear array if the system is operating in aone-axis scanning mode. This would be the case for HPO(horizontal-parallax-only) 3-D display, in which the system emits aswept set of vertical ray fans.

Illumination

Light sources 65 (shown in FIG. 3C) such as lasers, LEDs, or any othersuitable light source may be used within the scope of the presentinvention.

Alternatively, in accordance with embodiments of the present invention,imagery may be generated using techniques analogous tospatially-multiplexed autostereoscopic displays, e.g. lenticular arrayand integral photography (fly's eye lens array) display. In theseapproaches, each tile is associated with a lens positioned such thatillumination from the sub-pixels of the tile is directed towards a givenangle with respect to a normal of the tile. It should be recognized thata nanophotonic array may have several antennas producing light at thesub-pixel level. A benefit of this approach is that the sub-pixels canbe made smaller, using nanophotonic antenna(s), than traditional LCD orOLED pixels, thereby resulting in higher quality imagery.

Referring now to FIG. 13, a single tile 1400 of a display array isshown, configured as an element of a spatially multiplexed display,using active steering by a network of switches 1420. A series ofswitches 1420 direct incident light 1422 of an array of emitters towarda particular output 1425 of a nanophotonic array designated generally bynumeral 1430. Larger displays can be constructed by adjoining many tilesinto a surface, e.g., 1,024 tiles×768 tiles. A lenticular lens 1410serves the function, in such an embodiment, of the phase shifters orphase delay lines of the previous examples.

FIG. 14 illustrates an alternative steering technique that employspassive wavelength-selective switching, also as a tile 1510 of aspatially multiplexed display using a lenticular lens 1500. First, alaser is set to a first wavelength. Then data is fetched correspondingto the first view direction for the scene to be displayed by the displaycontroller. The light is gated into the filter, based on the value inthe frame buffer for that sub-hologram (e.g., if the pixel is to beperceived of as bright, the laser light should be allowed to pass intothe filter). Next, the wavelength of the laser is incremented. A pointerto the data from the scene is also incremented and the process is loopedfor each data point.

In this technique, the beam direction depends on the wavelength of lightas produced by the display controller. A wavelength filter 1520 directsthe light to a particular output, depending on the wavelength of thelight. In this case, the illumination wavelength would be changed toscan the beam using the display controller, and the light would berelatively narrowband, such as that from an external cavity or othertunable laser. Within the scope of the present invention, the colors maybe close together, relative to the ability or inability of a humanobserver to discriminate their relative hues. The light wavelengths areeach provided to a different pixel or subpixel location within thenanophotonic array of tiles 1510 and the light beam is directed at anormal angle from the tile wherein the lenticular nature of the lenscauses the light beam to be directed in the desired direction.

Methods are now described with reference to FIG. 15 and FIGS. 16A-16Bthat employ nanophotonic phased arrays to steer the light in each tile1500. In the phased array systems, the beam is formed by a combinationof light from an array of outputs of the tile 1500 (i.e., the tile iscomposed of a nanophotonic array that has multiple outputs), all ofwhich are active at once. A relative phase difference betweenneighboring emitters determines the direction of the beam. For example,if the relative phase difference is zero (all phases equal), the beam isdirected straight up with respect to FIG. 15. To direct the beam, aconstant phase difference is needed between successive elements. FIG. 15shows a distribution tree with phase shifters (1510 and 1515) (hereillustrated in a binary fashion) which can impart such a phase shift toeach element through splitter 1520A. Of the many arrangements todistribute power and such phase shifts, this arrangement has variousadvantages. One advantage is that only one control signal is necessaryfor all the phase shift devices because there is a constant relationshipbetween all the phase shifts (all phase shifts are multiples of a singleinput, Φ.) A second advantage is that it is possible to use thistechnique for relatively broadband light, such as that from an LED,because all path lengths are matched.

A passive method for phase array steering is now described withreference to FIG. 16A. In the passive phase array, designated generallyby numeral 1710, light follows a long path, shown as a serpentine path1703 in the figure. At equal spacings and at equal distances, some ofthe light is tapped out of the path and exits an output 1701. A coupler1704 is used to take a fraction of the light out of the path. This lightthen goes to the output 1701. As the wavelength changes, the phasedifference between these outputs 1701 changes in a relative manner,shifting the position of an emergent beam 1710 (shown in FIG. 16B). Theposition of the beam 1710 switches due to the relative phases of theoutputs, alone. The longer the path length between elements, the lesswavelength change is needed for steering. This structure is analogous toa grating, and could also be understood by considering the locations ofthe output couplers as elements of a grating. FIG. 16B shows anexemplary tile 1702 with multiple outputs 1701 in which the lightfollows the serpentine path as shown in FIG. 16A. Light is thus emittedfrom the output based upon the change in wavelength. The direction ofthe emitted light is determined by the wavelength. If there is a singlewavelength, then the light is emitted in a single direction.

For FIGS. 15 and 16A-16B, the methodology may occur as follows: firstset Φ=0 and the address pointer to ‘0’. Data from the data store of ascene is retrieved. Each sub-hologram is illuminated as a function ofthe data. The methodology then increments phi and the video datapointer. The process is the looped for each data element (e.g. pixelelement, sub-pixel element) until all of the data for the scene isprocessed.

As will be clear to those familiar with the arts of 3-D display andphased arrays, the techniques of the preceding section about active andpassive filter alternatives are illustrated in a mode suitable forsingle-axis beam scanning, which is referred to ashorizontal-parallax-only (HPO) operation in the field of 3-D display.The techniques can be extended to multi-axis (e.g. full parallax)scanning in a straightforward manner, by increasing the number ofelements and appropriately arranging the multiplexing or scanningelements.

The embodiments of the invention described above are intended to bemerely exemplary; numerous variations and modifications will be apparentto those skilled in the art. All such variations and modifications areintended to be within the scope of the present invention as defined inany appended claims.

What is claimed is:
 1. A radiation projector comprising: a plurality ofnanophotonic antennas configured to emit electromagnetic radiation; aplurality of phase delay elements, each phase delay elementcharacterized by a phase delay, at least one phase delay elementassociated with at least one of the plurality of nanophotonic antennas;and a control signal generator configured to generate a control signalassociated with the plurality of delay elements, wherein the controlsignal generator is further configured to recurrently update phasedelays of the plurality of phase delay elements in such a manner as tocause the electromagnetic radiation to substantially span at least onecontiguous solid angle.
 2. A radiation projector according to claim 1wherein an output of the plurality of nanophotonic antennas generates athree dimensional light field of imagery.
 3. A radiation projectoraccording to claim 1 further comprising: a modulator for receiving datarepresentative of a three dimensional scene scanned in several differentdirections and producing a plurality of time-varying illuminationpatterns.
 4. A radiation projector according to claim 3 wherein theelectromagnetic radiation emitted by the antennas is visible light.
 5. Aradiation projector according to claim 3 wherein the plurality ofnanophotonic antennas is coupled to the modulator for dividing thetime-varying illumination patterns into a plurality of paths, wherein apath is associated with each antenna within the array.
 6. A radiationprojector according to claim 5 wherein the plurality of delay elementsare a plurality of phase shifters each associated with an antenna forshifting the phase of the time-varying illumination patternscollectively so that the patterns are modulated in synchrony for each ofseveral directions.
 7. A radiation projector according to claim 1,wherein the phase delay elements shift the phase in a single direction.8. A radiation projector according to claim 1, wherein the phase delayelements shift the phase in both a horizontal and a vertical direction.9. A radiation projector according to claim 1 further comprising: acontroller for providing phase shift control signals to each of thephase delay elements to provide a sequence of video images.
 10. Aradiation projector according to claim 1, wherein the phase delayelements are phase shifters and the phase shifters are associated withan array of nanophotonic antennas that represent a single pixel and thephase shifters for the single pixel receive a phase control signal inthe x direction and in the y direction.
 11. A radiation projectoraccording to claim 1, wherein only two phase shift control values areneeded to steer a radiation pattern from an antenna.
 12. A radiationprojector according to claim 1 wherein the plurality of nanophotonicantennas are divided into a plurality of arrays and each arrayrepresents a single pixel.
 13. A radiation projector according to claim1 wherein the plurality of nanophotonic antennas are divided into aplurality of arrays and each array represents a single hogel.
 14. Aradiation projector according to claim 1 wherein the plurality ofnanophotonic antennas are divided into a plurality of arrays and eacharray represents a single wafel.
 15. A radiation projector according toclaim 1 wherein the plurality of phase delay elements is a MEMS phaseshifter that mechanically alters phase based upon movement of a phaseactuator.
 16. A radiation projector according to claim 15 wherein thephase actuator is a membrane.
 17. A radiation projector according toclaim 1 further comprising: a database containing the datarepresentative of the three dimensional scene scanned from severaldifferent directions.
 18. A radiation projector according to claim 1wherein the emitted output radiation can be directed to converge at aspatial region, such that a viewer can view the emitted output radiationif the viewer is within the spatial region and the viewer cannot viewthe emitted output radiation if the viewer is outside of the spatialregion.
 19. A radiation projector according to claim 1, wherein thenanophotonic antennas are controlled with a first control signal tosteer the emitted output radiation in a first direction.
 20. A radiationprojector according to claim 18 wherein the nanophotonic antennas arecontrolled with a second control signal to steer the emitted outputradiation in a second direction.
 21. A radiation projector according toclaim 1, wherein the plurality of nanophotonic antennas are arranged inan array and the array is arranged on a two dimensional surface.
 22. Aradiation projector comprising: a lens; and a plurality ofinterconnected switches having an input and an output wherein the inputreceives a time-varying illumination pattern and the time-varyingillumination pattern is synchronized with control signals to theplurality of switches allowing the time-varying pattern to be directedin a desired direction through the output of the switches and throughthe lens, wherein each switch has an associated state.
 23. The radiationprojector of claim 21 wherein the plurality of interconnected switcheshas a plurality of outputs that defines a pixel, wherein the directionof the emitted time-varying illumination pattern is dependent on thestate of the switches.
 24. A radiation projector comprising: alenticular lens; an input for receiving a time-varying illuminationpattern; a nanophotonic array having a plurality of outputs; and afilter for directing the time-varying illumination pattern to aparticular output of the nanophotonic array so that the time-varyingillumination from the output is directed to the lenticular lens.
 25. Amethod for generating a three-dimensional radiation pattern, the methodcomprising: receiving data from a data store that defines a threedimensional image; converting the data into a time-varying illuminationpattern and providing the time-varying illumination pattern to an inputof a nanophotonic array, wherein the nanophotonic array includes aplurality of antennas and the antennas emit electromagnetic radiation;and delaying the electromagnetic radiation with a plurality of phasedelay elements using a control signal, at least one phase delay elementassociated with at least one of the plurality of antennas, wherein thecontrol signal is periodic.
 26. A method according to claim 25, whereinreceiving data from a data store includes receiving data representativeof a three dimensional scene scanned in several different directions andproducing a plurality of time-varying illumination patterns.
 27. Amethod according to claim 25, wherein an output of the plurality ofantennas generates a three dimensional light field of imagery.
 28. Amethod according to claim 25, further comprising receiving datarepresentative of a three dimensional scene scanned in several differentdirections and producing a plurality of time-varying illuminationpatterns.
 29. A method according to claim 25, wherein the step ofdelaying includes shifting the phase of the time-varying illuminationpattern so that the pattern is modulated in synchrony with timinginformation from the data.
 30. A method for generating athree-dimensional radiation pattern, the method comprising: receivingdata from a data store that defines a three dimensional image;converting the data into a time-varying illumination pattern andproviding the time-varying illumination pattern to an input of ananophotonic array wherein the nanophotonic array includes a pluralityof antennas and the antennas emit electromagnetic radiation; andswitching the electromagnetic radiation with a plurality of phase delayelements using a control signal, at least one switching elementassociated with at least one of the plurality of antennas, wherein thecontrol signal is periodic.
 31. A method according to claim 30, whereinreceiving data from a data store includes receiving data representativeof a three dimensional scene scanned in several different directions andproducing a plurality of time-varying illumination patterns.
 32. Amethod according to claim 30, wherein an output of the plurality ofantennas generates a three dimensional light field of imagery.
 33. Amethod according to claim 30, further comprising receiving datarepresentative of a three dimensional scene scanned in several differentdirections and producing a plurality of time-varying illuminationpatterns.
 34. A method according to claim 30, wherein the step ofdelaying includes shifting the phase of the time-varying illuminationpattern so that the pattern is modulated in synchrony with timinginformation from the data.
 35. A radiation projector comprising: aplurality of optical couplers, each optical coupler transmitting a firstportion of a lightwave incident thereupon and radiating a second portionof the lightwave, the lightwave characterized at any point by awavelength-dependent phase; and a waveguide for transmitting thelightwave successively to a succession of the plurality of opticalcouplers in such a manner that the wavelength-dependent phase variesbetween successive optical couplers by a fixed wavelength-dependentincrement.